Wire garak into the Arsenal: structured parser + report-file capture - #128
Wire garak into the Arsenal: structured parser + report-file capture#128TengHu wants to merge 1 commit into
Conversation
Turn garak's report.jsonl into structured ToolFinding[] and feed the report FILE (not stdout) to the parser, so ai_red_team missions produce verified, tool-backed findings instead of unverifiable prose. - parsers.ts: parseGarak() aggregates attempts into one finding per (probe x detector) with attack-success-rate + severity; registered in PARSERS. Summarises garak's own detector verdicts; never emits prompt/ response transcript text (names + counts only). - catalog.ts: garak parserStatus planned -> structured. - adapter-tools.ts: ArgTemplate.reportFile hook + optional mkReportPath/readToolReport deps + garak template. The handler reads a report-file tool's output from disk and uses it as BOTH the parse input and the evidence of record. Zero behaviour change for stdout tools. - index.ts: real deps — temp-path minter + consume-then-delete report reader (probe transcripts do not linger on disk). - tests: garak fixture + field-mapping, aggregation, no-transcript-leak, honesty-contract, gate-pass, and a factory test proving the report file (not stdout) is parsed. Updated the invocation-honesty guard (garak is now templated).
jmagly
left a comment
jmagly
left a comment
There was a problem hiding this comment.
The exact head passes typecheck, lint (warnings only), and the full 652-test suite, and the structured parser itself looks sound. One security/privacy blocker remains: garak report JSONL can contain prompt/response transcripts, but it is created directly in the shared system temp directory and cleanup only happens after a successful read. A timeout, spawn error, non-zero garak exit, or read failure returns before deletion, leaving sensitive transcripts behind; direct /tmp creation also does not establish a private 0700 parent directory.
Please create reports inside a private per-run directory (for example mkdtemp with restrictive permissions) and remove the report/directory in a finally path covering success, non-zero exit, timeout, spawn error, and read failure. Add regression coverage for failure-path cleanup and the private temp location/permissions.
2 participants
What & why
garak was catalogued but
parserStatus: 'planned'— an agent could run it, but itsoutput could only enter as the model's prose, which the live provenance gate
(
src/evidence/gate.ts) refuses to mark verified. So theai_red_teamfamily hadno verified, tool-backed evidence source.
This wires garak end-to-end:
parseGarak(src/arsenal/parsers.ts) — turns garak'sreport.jsonlintoToolFinding[], one finding per(probe × detector)that registered an attacksuccess, with attack-success-rate → severity. It summarises garak's own detector
verdicts (a parser summarises, it does not re-judge) and surfaces only probe/
detector names + counts — never prompt/response transcript text.
src/arsenal/adapter-tools.ts+src/index.ts) — garakwrites its results to a
report.jsonlfile, not stdout. An additiveArgTemplate.reportFilehook + optionalmkReportPath/readToolReportdeps let thehandler read that file back and use it as both the parse input and the evidence
of record. Zero behaviour change for every stdout-based adapter.
catalog.ts: garakparserStatus: 'planned' → 'structured'.Result: an
ai_red_teammission can run garak against an authorized model and producefindings that pass the provenance gate
Scope honesty (please read)
ground-truth-verified ("the model is really jailbroken"). garak's keyword detectors
(
dan.DAN,mitigation.MitigationBypass,TriggerListDetector) over-report — inlocal testing on a small model, a majority of
LatentJailbreak"hits" were refusals(false positives). The finding
detailssays so and recommends transcript verification.An LLM-judge detector is the natural follow-up (separate PR).
Contribution Receipt
parseGarak+ report-file capture, soai_red_teammissions produce verified, tool-backed findings instead of unverifiable prose.static_fixturenot_applicable(tests run against a committed static fixture; no live target in this change)nonestatic_test,tool_backednot_applicablefor the change's tests. The committed fixture is derived from a real garak 0.15.1 run againstmeta-llama/llama-3.2-3b-instructvia OpenRouter/litellm (author's own lab; not part of CI).npm run typecheck(tsc --noEmit) → pass.npm test(fullvitest run src) → 651/652 pass. The single failure —local-agent-path-resolution.test.ts(codexinstall detection, issue #78) — reproduces identically on the unmodified baseline (verified by stashing this change) and is environmental: the author's machine has thecodexCLI installed, which the temp-home test does not expect. It is unrelated to this change and does not touch any file in it.eslint(changed files) → clean.src/__tests__/fixtures/garak.report.jsonl(committed fixture).readToolReportconsume-then-deletes the report file so probe transcripts do not linger on disk. Fixture contains no secrets.none— no README/headline claim changed; garak is not added to any benchmark.not_applicable(no model behaviour measured in this change).promptfooremainsplanned.Test evidence
parseGarakfield-mapping, aggregation, honesty-contract (empty/garbled →[]), andno-transcript-leak assertions.
gateLiveFindingwithprovenance: 'tool'.